Visualization of images via an enhanced eye tracking system

ABSTRACT

In this patent, an improved eye tracking system is implemented for enhanced viewing. This system incorporates eye facing cameras and tracks display settings longitudinally and determines which portions of the image have been viewing and to what extent. A longitudinal dataset is generated and analyzed to better understand the human review process and cause improvements thereof. Furthermore, image review is enhanced by highlighting portions of the image that have not been adequately reviewed. Comparison across multiple reviewers is also performed to improve a user&#39;s performance.

CROSS REFERENCES TO RELATED APPLICATIONS

This patent application is a continuation-in-part of U.S. patentapplication Ser. No. 16/879,758 filed on May 21, 2020 and U.S. patentapplication Ser. No. 16/842,631 filed on Apr. 7, 2020. In addition, thispatent application claims the benefit of U.S. Provisional PatentApplication 62/856,185 filed on Jun. 3, 2019, U.S. Provisional PatentApplication 62/985,363 filed on Mar. 5, 1920 and U.S. Provisional PatentApplication 62/939,685 filed on Nov. 25, 2019.

TECHNICAL FIELD

Aspects of this disclosure are generally related to image processing.

INTRODUCTION

Many occupations rely on work with a computer and utilize imageprocessing. For example, an air traffic controller, a radiologist, asoftware engineer and others.

SUMMARY

All examples, aspects and features mentioned in this document can becombined in any technically conceivable way.

The primary purpose of this patent is to improve a radiologist's abilityto analyze images through incorporation of eye tracking. Morespecifically, a radiologist's eyes will be tracked with an eye trackingcamera. A variety of eye tracking systems can be used. For example, thehead mounted Eyelink II or remoteEyeLink 1000 eye tracking system (SRResearch Ltd., Ottawa, Ontario, Canada), sampled monocularly at 500 Hz.Error is within 1 degree. Example system parameters include:acceleration threshold of 9000 degrees per sec²; velocity threshold of30 degrees per sec; and deflection threshold of 0.1 degree. The metricsfrom the eye tracking camera will be utilized to optimize the image. Thedetails of this process are disclosed herein.

In summary, this patent teaches a method comprises displaying an imagingdataset on a monitor to a user. Then, perform segmentation of theimaging dataset into discrete imaging features. This can be done byconventional segmentation tools (e.g., for the brain, the FreeSurfertool can be implemented for segmentation for brain datasets). Then,determine the location(s) of imaging features on the monitor. Then,track eye movements of a user to determine the fixation locations atpixels on the monitor and the corresponding imaging features beingviewed. It is important to note that a key step in this process isadjusting for image panning and zooming. This is because the image isnot on a fixed location on the screen. Therefore, tracking of a user'seyes to determine a fixation location on a monitor, tracking of theimage location on the monitor and tracking the image size on the monitorare performed. Then, record data on fixation locations and discreteimaging features. Then, analyzing the recorded data on fixationlocations and discrete imaging features.

In order to optimize the image being viewed based on eye trackingmetrics, a longitudinal dataset is generated. Metrics correlatingfixation locations with imaging features are created. First, one metricis whether or not an imaging feature has at least one fixation location.For example, it would be important to have at least one fixationlocation on the optic nerve insertion on the eyeball because this is acritical area of importance to a radiologist. Additionally, thepituitary stalk is a small structure less than 3 mm wide (in thetransverse direction). This structure, while small, is important becauseit can harbor a variety of pathologies. Thus, this technique ofdetermining whether the pituitary stalk has at least one fixationlocation is useful. Next, it would be important to have the number offixation locations for an imaging feature a metric. For example, aminimum number of fixation points should be on the kidney is correlatedto the minimum adequate review. Additionally, if a particular imagingfeature has an excess number, then it could be an indicator forpathology. For example, if the right optic nerve has been viewed with 50fixation locations, but the average number of fixation locations for apopulation of imagers is 3 with a standard deviation of 2, then it wouldbe determined that the particular imaging feature has been viewed anexcess number of times and this could be an indicator of pathology. Thisindicator can be used further by AI/machine learning processes as well.For example, the length of time of each fixation location is alsocorrelated to the minimum adequate review. In addition, the length oftime of each fixation location is also a useful metric. For example,typical fixation duration are on the order of ˜200 msec. If a singlefixation duration is found to be on the order of ˜800 msec or longer,then it could be flagged as a spot that is at a higher risk of harboringpathology. Next, the location within an imaging feature of each fixationlocation is a metric. It is not enough to document a single fixationpoint in the pituitary gland, a fixation location on the small posteriorpituitary bright spot is also important. For example, common spotswithin the lateral ventricle on a CT scan of the head include theposterior aspects of the occipital horns. If an atypical location wasviewed (e.g., the central portion of the frontal horn of the lateralventricle well away from the ependymal lining), then this imagingfeature can be flagged as higher risk and more likely to harborpathology. Next, an adequate review demands that a radiologist review animaging feature on multiple imaging planes (e.g., common bile duct);therefore, the number of imaging planes an imaging feature has fixationlocations in situations comprising wherein the imaging dataset comprisescross-sectional imaging planes is an important metric. For example, ifthat same spot in the frontal horn of the lateral ventricle were viewedon all three planes in an excessive fashion, then that spot may bedetermined to be high risk. For example, the sequence of fixationlocations and viewing of imaging features is also important, as thisindicate a search pattern of cause and effect. For example, if aradiologist were to notice a skull fracture, the radiologist should do adeliberate search for an epidural hematoma. These such metrics areutilized in accordance with U.S. patent application Ser. No. 16/842,631,A SMART SCROLLING SYSTEM, which is incorporated by reference.Additionally, a typical sequence could be oscillating medial-lateral andmoving in an inferior fashion down the right lung and then oscillatingmedial-lateral and moving in a superior fashion up the left lung. Ifthis sequence was maintained, that could be an indicator that the studyis normal. If this sequence was broken, it could indicate that there isan abnormality within the image. Alternatively, it could indicated thatthere was an interruption (e.g., phone call). Finally, time of reviewper segmented structure and total time are additional metrics recorded.

In some embodiments, comparing the longitudinal dataset with apredetermined criteria to determine extent of review. Examples ofpredetermined criteria include, but are not limited to, the following: aminimum number of fixation locations for the imaging dataset; a minimumnumber of fixation locations for each imaging feature; a minimum numberof fixation locations for each subsegmented area (e.g., head of thepancreas) within an imaging feature (e.g., pancreas); a minimum time offixation location for each imaging feature; a minimum number of imagingplanes an imaging feature has fixation locations in situationscomprising wherein the imaging dataset comprises cross-sectional imagingplanes; whether or not the imaging structure had optimized displayduring a fixation location; and whether or not a predetermined sequenceof fixation locations for each imaging feature has been achieved.

In some embodiments, data is averaged from each metric over a set ofimaging datasets to determine an average for each metric for the user.For example, the same metrics can be obtained on the same user forreview of 100 non-contrast head CT exams. In some embodiments, anotification to the user is performed when a metric for the imagingexamination differs from the average for each metric for the user. Forexample, assume that the average time period for the user to look at themidline sagittal image of the brain was 10.5 seconds and assume thatthere was no case in the past 100 cases where the user had viewed themidline sagittal image for less than 8 seconds. Assume that the userviewed the midline sagittal image for 0.1 seconds. That would be asignificant deviation of the normal. It could be related to a variety ofuser errors (e.g., phone call). The method would alert the user of thisanomaly and allow for the opportunity to correct the error.

Some embodiments further comprising averaging data from each metric overa set of imaging datasets to determine an average for each metric for apopulation of users to develop a set of normative metrics. Comparison ofa user's metrics to a population dataset can be performed to bring outdeviations from the normal. As a result, the user can learn and improve.

The preferred embodiment is a method of altering display settings basedon information from eye-tracking technology. A list of optimal viewingsettings for each item in an image is generated. The item that islocated at each viewing location (e.g., pixel on 2D monitor or 3D pointin space corresponds to liver) is determined. An eye-tracker system withat least one eye-facing camera is initiated. Eye-movement data with thesaid eye-facing camera(s) is recorded. Analysis of the eye-movement datato determine where the user is looking (e.g., the focal point) isperformed. Analysis of where the user is looking to determine whichobject the user is examining is performed. The current image viewingsettings with optimal viewing settings for the item in an image beingexamined is performed to determine whether the viewed object isoptimally displayed to the user. If the viewed object is alreadyoptimally displayed to the user, no changes to the image would beperformed. If the viewed object is not already optimally displayed tothe user, manipulate image such that it is optimally displayed. Finally,eye tracking and optimization of display settings is continued. The eyetracking techniques can cause 2D datasets (e.g., chest radiograph) or 3Ddatasets (e.g., Computed Tomography scan slices) to be optimized. Theeye tracking techniques can be performed in conjunction with 2D displays(e.g., conventional 2D radiology monitors), advanced curved monitors(single curve or double curved) or extended reality head displays.

Some embodiments comprise altering the displayed image based on therelationship between the analyzed data and the predetermined criteria.An example includes altering the brightness of imaging feature(s) basedon whether or not the predetermined criteria for the imaging feature hasbeen met. For example, an imaging feature(s) that has been met thepredetermined threshold is assigned a first visual representationadjustment logic (dark shades of gray) and an imaging feature(s) thathave not met the predetermined threshold is assigned a second visualrepresentation adjustment logic (bright shades of gray).

Some embodiments comprise providing visual feedback to the user onpredetermined criteria to assure that the user performs a comprehensivereview of the imaging dataset. Examples include, but are not limited to,the following: circles; annotations; and, arrows.

Some embodiments comprise performing eye tracking, which causes someareas within the image to be viewed to be displayed with variabledisplay settings. Examples of the user preferred display settingsinclude: windowing and leveling of at least one portion of an image;kernel (e.g., bone kernel to soft tissue kernel) of at least one portionof an image; color of at least one portion of an image; voxelprioritization, as described in U.S. patent application Ser. No.16/879,758, A METHOD AND APPARATUS FOR PRIORITIZED VOLUME RENDERING ofat least one portion of an image; band-wide grouping of at least oneportion of an image, as described in U.S. Pat. No. 10,586,400,PROCESSING 3D MEDICAL IMAGES TO ENHANCE VISUALIZATION, which isincorporated by reference; filtering of at least one portion of animage; latency (how long the user should look at an object prior to theimage display settings being manipulated); technique on switching to newimage display settings (fading in the new image settings slowly orimmediately in a subsequent frame show the new image).

Some embodiments comprise utilization of inputs from a user's hands(e.g., via controller, keyboard, hand gestures, etc.) to guide/overridewhether or not to change settings to a more optimal viewing can be basedon eye position. Examples of the settings are previously described.

In some embodiments, the cameras assess the user's face for facialexpressions, which may be indicative of a lesion. If such a facialexpression is identified, then this can be brought to the attention ofthe user (e.g., radiologist) during report generation.

Given that the majority of radiology departments still use flat screenmonitors, this patent's process is anticipated to be first used onexisting monitors. Eye-facing cameras can be incorporated. It isenvisioned that future embodiments comprise wherein the monitor has acurvature comprising: the top portion of the monitor curves inwardstowards the user; the bottom portion of the monitor curves inwardstowards the user; the left portion of the monitor curves inward towardsthe user; and the right portion of the monitor curves inward towards theuser. A wide range of images can be displayed on this “double curved”monitor to enhance viewing. Examples include conventional viewing ofradiological images and advanced viewing techniques as disclosed in thispatent. Some embodiments further comprise utilizing a head display unit(HDU) comprising at least one of an extended reality display, shutterlenses and polarized lenses wherein the HDU provides a 3D image to theuser. Furthermore, more ergonomic keyboards are also utilized with themiddle of the keyboard elevated as compared to the sides so as to reducethe required pronation during typing.

Some embodiments comprise a monitor-image conversion key. In thisembodiment, a monitor coordinate system is established. Then, at eachtime point in the dataset, at least one data point within an imagewherein the at least one data point within an image serves as areference point for all other data points within the image. Furthermore,at each point in the dataset, a recording the zoom state of the image isperformed. Additionally, eye tracking is performed to determine a user'sfixation location on the monitor coordinate system. Using themonitor-image conversion key, which data point within the imagecorresponds to which fixation location can be determined.

Some embodiments comprise assigning a set of predetermined locationswithin an image that should be viewed by a user in order for acomprehensive review to be performed. For example, the pituitary stalkcould be a predetermined location that should be viewed. In the eventthat the user has the minimum number of fixation points on the pituitarystalk, this criteria would be satisfied. However, if the predeterminedlocation (e.g., pituitary stalk) was not visualized, options includecreating a reminder (e.g., stating “look at pituitary stalk”),presenting the predetermined location (e.g., sagittal midline imageshowing the pituitary stalk), or combination thereof.

Some embodiments comprising wherein the user is alerted of thosepredetermined locations which have not been viewed by at least one ofthe group comprising: providing a visual alert cue adjacent to thosepredetermined locations which have not been viewed; and providing afirst visual representation adjustment logic for the pixels nearby thepredetermined locations which have been viewed and a second visualrepresentation adjustment logic for those pixels nearby thepredetermined locations which have not been viewed. Thus, this systemmay incorporate imaging findings on a radiologist's checklist.

Some embodiments incorporate digital objects in proximity to apredetermined location that has not been viewed, so as to draw theuser's attention to the predetermined location and provide a morecomprehensive view.

Some embodiments comprise using artificial intelligence algorithms tounderstand the correlation between eye movements and pathologicalconditions.

Some embodiments comprise performing “smart panning” based on eyetracking. In this embodiment, eye tracking data can cause the displayedimage to pan to a new location. For example, if the user is looking at aparticular region, the “smart panning” function can be implemented topan the image to a new location. For example, if the user is looking ata finding with numerous fixation locations (e.g., meets a predeterminedcriteria of number of fixation spots) on an imaging finding (e.g.,humerus bone on a chest x-ray) within a predetermined location (e.g.,distance to the edge of the monitor is less than 1 inch), then the smartpanning function can be implemented. This can be done automatically andthe entire image can be moved inward from the edge of the monitor sothat the user can see it better.

Some embodiments comprise performing “smart zooming” based on eyetracking. In this embodiment, eye tracking data can cause the displayedimage to zoom to a new size (smaller or bigger). For example, if theuser is looking at a very tiny structure (e.g., brain aneurysm), smartzooming can enlarge the image automatically to the optimized size.

Some embodiments comprise performing “smart window/level” based on eyetracking. In this embodiment, eye tracking data can cause the imagedisplayed in a fashion that is optimized for the structure being viewed.For example, if the user is looking at the liver, smart window/level canautomatically display the liver in a liver window.

Some embodiments comprise wherein the image display setting is changeddynamically. For example, the user changes the window/level setting,scrolls through the liver, zooms, pans, changes the liver again. Duringthis time, eye tracking data is being acquired to determine whichstructures the user is looking at the whole time and what displaysetting is shown.

Some embodiments comprise recording pupil size to determineaccommodation and incorporating these data into the longitudinaldataset.

Some embodiments comprise free-viewing as no external stimuli of whatimaging features should be looked at. In this embodiment, predeterminedcriteria can be implemented on eye movements, mouse movements,window/level settings and zoom settings, so as to assure that certainfeatures are optimized. As previously discussed, if a predeterminedcriteria is not met during the free-viewing, the user can be alerted.

Some embodiments comprise a guided-viewing process, as external stimuliof what imaging features should be looked at. An example of the guidedviewing process is automatic scrolling, automatic windowing/leveling,automatic panning, automatic zooming and use of digital objects to guidea user to look at certain spots in the image. A digital dot could be inthe form of one or more small objects on the screen. Alternatively, thecursor could automatically move and be used as a digital object to guidethe radiologist in viewing of images.

Some embodiments comprise showing a digital object at the location of auser's fixation locations. The key application of this embodiment is ateaching environment. For example, a radiology attending can watch aradiology resident's fixation locations on the screen in real time. Suchan option could be performed while using augmented reality headsets. Forexample, user #1 could see where user #2 is looking, but user #1 wouldnot see his own digital object (which would be a distraction). In someembodiments, a head tracking system could also be utilized in a similarfashion to perform optimization of the imagery.

Other arrangements of embodiments of the invention that are disclosedherein include software programs to perform the method embodiment stepsand operations summarized above and disclosed in detail below. Moreparticularly, a computer program product is one embodiment that has acomputer-readable medium including computer program logic encodedthereon that when performed in a computerized device provides associatedoperations providing three-dimensional viewing of images by a user asexplained herein. The computer program logic, when executed on at leastone processor with a computing system, causes the processor to performthe operations (e.g., the methods) indicated herein as embodiments ofthe invention. Such arrangements of the invention are typically providedas software, code and/or other data structures arranged or encoded on acomputer readable medium such as an optical medium (e.g., CD-ROM),floppy or hard disk or other a medium such as firmware or microcode inone or more ROM or RAM or PROM chips or as an Application SpecificIntegrated Circuit (ASIC) or as downloadable software images in one ormore modules, shared libraries, etc. The software or firmware or othersuch configurations can be installed onto a computerized device to causeone or more processors in the computerized device to perform thetechniques explained herein as embodiments of the invention. Softwareprocesses that operate in a collection of computerized devices, such asin a group of data communications devices or other entities can alsoprovide the system of the invention. The system of the invention can bedistributed between many software processes on several datacommunications devices, or all processes could run on a small set ofdedicated computers, or on one computer alone.

It is to be understood that the embodiments of the invention can beembodied strictly as a software program, as software and hardware, or ashardware and/or circuitry alone, such as within a data communicationsdevice. The features of the invention, as explained herein, may beemployed in data processing devices and/or software systems for suchdevices.

In some embodiments, viewing of a radiology image is performed on amonitor wherein the top portion of the monitor curves inwards towardsthe user, the bottom portion of the monitor curves inwards towards theuser, the left portion of the monitor curves inward towards the user;and the right portion of the monitor curves inward towards the user. Insome embodiments, an eye-facing camera(s) is used in conjunction withthis monitor, such that eye-tracking can be performed in conjunctionwith these techniques. In some embodiments, a computer is used inconjunction with the eye tracking. In some embodiments, the computerperforms an algorithm to record areas or volumes which have beenreviewed and areas or volumes which have not been reviewed. In someembodiments, the computer displays areas or volumes which have beenreviewed differently from areas or volumes which have not been reviewed.In some embodiments, techniques described in U.S. Pat. No. 10,586,400,which is incorporated by reference.

Note that each of the different features, techniques, configurations,etc. discussed in this disclosure can be executed independently or incombination. Accordingly, the present invention can be embodied andviewed in many different ways. Also, note that this summary sectionherein does not specify every embodiment and/or incrementally novelaspect of the present disclosure or claimed invention. Instead, thissummary only provides a preliminary discussion of different embodimentsand corresponding points of novelty over conventional techniques. Foradditional details, elements, and/or possible perspectives(permutations) of the invention, the reader is directed to the DetailedDescription section and corresponding figures of the present disclosureas further discussed below.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates the overview of a smart review process.

FIG. 2 illustrates an overview of the apparatus used for eye tracking indiagnostic radiology.

FIG. 3A illustrates a top down view is shown and the user is looking ata pixel on the left aspect of the screen.

FIG. 3B illustrates a side view wherein the user is looking at a pixelnear the top of the screen.

FIG. 4A illustrates an eye-tracker system with an eye-facing camera(s)working in conjunction with an extended reality headset.

FIG. 4B illustrates an eye-tracker system an extended reality headsetwith an on-board with eye-facing camera(s).

FIG. 5 illustrates fixation points illustrated on a CT scan slicethrough the upper abdomen identified by the eye-tracking system.

FIG. 6A illustrates a side view of the radiologist's workstation.

FIG. 6B illustrates a top-down view of the radiologist's workstation.

FIG. 7A is a top view of a monitor screen.

FIG. 7B is a front view of the monitor screen shown in FIG. 7A.

FIG. 7C is a rear view of the monitor screen shown in FIG. 7A.

FIG. 7D is the right side view of the TV/monitor screen shown in FIG.7A.

FIG. 7E is the left side view of the TV/monitor screen shown in FIG. 7A.

FIG. 7F is the bottom view of the TV/monitor screen shown in FIG. 7A.

FIG. 7G is a cross-sectional view taken along line A in FIG. 7B.

FIG. 7H is a cross-sectional view taken along line B in FIG. 7B.

FIG. 8A is a view from the top of a keyboard (keys not shown) lookingdown.

FIG. 8B is a cross-section of the keyboard.

FIG. 9A illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a first location on the monitor.

FIG. 9B illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a second location on the monitor.

FIG. 10A illustrates the monitor coordinates and first coordinates foran image.

FIG. 10B illustrates the monitor coordinates and second coordinates foran image.

FIG. 11A illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a first location and a first zoom status on the monitor.

FIG. 11B illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a second location and a second zoom status on the monitor.

FIG. 12 illustrates application of the conversion key to convert where auser is looking on a computer monitor to where a user is looking on animage.

FIG. 13 illustrates generation longitudinal dataset.

FIG. 14 illustrates a method of altering display settings based oninformation from eye-tracking technology.

FIG. 15 illustrates an example wherein the spot at which the user islooking already has optimized viewing settings and no changes are made.

FIG. 16 illustrates an example wherein the spot at which the user islooking is not currently optimized viewing settings and changes to theviewing settings are automatically performed.

FIG. 17A illustrates assigning zones wherein when a user looks at aparticular zone, a corresponding image manipulation occurs.

FIG. 17B illustrates dividing up the field of view into regions based onsegmentation.

FIG. 18 illustrates generating a list of the optimal viewing settingsfor each item in an image.

FIG. 19A illustrates an image optimized for visualization of abdominalorgans.

FIG. 19B illustrates an image optimized for visualization of abdominalorgans with a fixation point.

FIG. 19C illustrates an image, which has been altered due to a priorfixation point.

FIG. 19D illustrates an image, which is partially darkened with a secondfixation point shown.

FIG. 19E illustrates an image, which is partially darkened at thelocations of the two prior fixation points.

FIG. 19F illustrates an image, which is completely darkened, whichoccurs when a slice is fully inspected.

FIG. 20 illustrates an example of changing of the appearance of an imagein accordance with eye tracking.

FIG. 21 illustrates a smart zoom process.

FIG. 22A illustrates an imaging finding displayed on a monitor.

FIG. 22B illustrates an imaging finding displayed on a monitor.

FIG. 23A illustrates determining and recording which areas (e.g., voxelsor pixels) are included in the high resolution field of view (FOV),which areas (e.g., voxels or pixels) are included in the mediumresolution FOV, and which areas (e.g., voxels or pixels) are included inthe low resolution FOV.

FIG. 23B illustrates a zoomed in 8×8 set of voxels, showing assigningsome voxels to a high resolution FOV and some voxels to a mediumresolution FOV in accordance with FIG. 23A.

FIG. 24 illustrates a method of generating metrics based on whichimaging features have been reviewed.

FIG. 25 illustrates assigning a set of predetermined locations within animage that should be viewed by a user in order for a comprehensivereview to be performed.

FIG. 26 illustrates comparing the analyzed recorded data on fixationlocations and discrete imaging features with a predetermined criteria ofminimum imaging metrics that must be met for a complete review.

DETAILED DESCRIPTIONS

FIG. 1 illustrates the overview of a smart review process. Processingblock 100 illustrates the step of preparing the dataset (e.g., performsegmentation of the imaging dataset so that the image is segmented intovarious imaging findings, perform desired visual representationadjustment logic including the set of display settings (how the image isdisplayed, such as window/level settings), and display image(s).Processing block 101 illustrates the step of perform head and eyetracking and determine the pixel (or pixels) on the monitor (e.g., whichspecific monitor coordinates) where the user is looking and additionalmetrics (length of time, number of fixation points, etc.). Processingblock 102 illustrates the step of determining which imaging findingsuser is looking at (e.g., use monitor-image conversion key to accountfor zoom status and pan status and correlating the fixation location onthe monitor to the imaging feature). Processing block 103 illustratesthe step of recording data in longitudinal dataset along with otherimaging features (e.g., mouse location, set of display settings,determine which anatomic structures are being examined by eye tracking,which structure is actively being studied on the radiologist'schecklist, determine the length of time which anatomic structures arebeing examined, other viewing parameters, etc.). Processing block 104illustrates the step of analyzing the data and providing feedback to theuser (e.g., display a modified image, such as scroll to a contiguousslice for slice-by-slice review, zooming, panning, changing window andlevel settings, changing transparency for D3D review, etc.). Note thatthe modification of the image is also recorded in the longitudinaldataset.

FIG. 2 illustrates an overview of the apparatus used for eye tracking indiagnostic radiology. A radiologic imaging system 200 (e.g., X-ray,ultrasound, CT (computed Tomography), PET (Positron EmissionTomography), or MM (Magnetic Resonance Imaging)) is used to generatemedical images 202 of an anatomic structure 204 of interest. The medicalimages 202 are provided to an image processor 206, that includesprocessors 208 (e.g., CPUs and GPUs), volatile memory 210 (e.g., RAM),and non-volatile storage 212 (e.g. HDDs and SSDs). A program 214 runningon the image processor implements one or more of the steps described inthis patent. Medical images and displayed on an IO device 216, whichincludes an eye tracking system. The IO device may also include avirtual or augmented reality headset, monitor, tablet computer, PDA(personal digital assistant), mobile phone, or any of a wide variety ofdevices, either alone or in combination. The IO device may include atouchscreen and, may accept input from external devices (represented by218) such as a keyboard, mouse, and any of a wide variety of equipmentfor receiving various inputs. However, some or all the inputs could beautomated, e.g. by the program 214.

FIG. 3A illustrates a top down view is shown and the user is looking ata pixel on the left aspect of the screen. 300 illustrates the eyes of auser. 301 illustrates cameras that perform eye tracking. 302 illustratesthe monitor. 303 illustrates a pixel on the screen that a user islooking at. 304 illustrates light rays traveling from the user's eyes,which travel towards the cameras 301. 305 illustrates light raystraveling from the pixel on the screen that the user is looking attowards the user's eyes 300. The eye tracking system determines whichpixel the user is looking at.

FIG. 3B illustrates a side view wherein the user is looking at a pixelnear the top of the screen. 300 illustrates the eyes of a user. 301illustrates cameras that perform eye tracking. 302 illustrates themonitor. 303 illustrates a pixel on the screen that a user is lookingat. 304 illustrates light rays traveling from the user's eyes, whichtravel towards the cameras 301. 305 illustrates light rays travelingfrom the pixel on the screen that the user is looking at towards theuser's eyes 300. The eye tracking system determines which pixel the useris looking at.

FIG. 4A illustrates an eye-tracker system with an eye-facing camera(s)working in conjunction with an extended reality headset. The cameras useeye tracking and head tracking to determine that the user is gazing at a3D point within a kidney. Therefore, the display is optimized forviewing the kidneys. This optimization includes, but is not limited to,the following: conventional gray-scale optimization (e.g., windowing andleveling); double windowing in accordance with U.S. Pat. No. 10,586,400;and prioritized volume rendering in accordance with U.S. patentapplication Ser. No. 16/842,631. Also, the algorithm (per userpreference) states if the user is looking at one kidney, then displayboth kidneys with optimal display configurations. 400 illustrates anextended reality head display unit. 401 illustrates cameras whichperform eye tracking. 402 illustrates the virtual image of the kidney,which the user is looking at. 403 illustrates the convergence pointwhere the user is looking at within the kidney. 404 illustrates the lineof sight from the left eye to the convergence point 403. 405 illustratesthe line of sight from the right eye to the convergence point 403.

FIG. 4B illustrates an eye-tracker system an extended reality headsetwith an on-board with eye-facing camera(s). 406 illustrates an extendedreality head display unit, which contains eye tracking cameras on boardthe extended reality head display unit. 407 illustrates an eye trackingcamera for the left eye. 408 illustrates an eye tracking camera for theright eye. 409 illustrates the line of sight from the left eye to aconvergence point 412. 410 illustrates the line of sight from the righteye to the convergence point 412. 411 illustrates the virtual image ofthe kidney, which the user is looking at. 412 illustrates theconvergence point where the user is looking at within the kidney.

FIG. 5 illustrates fixation points illustrated on a CT scan slicethrough the upper abdomen identified by the eye-tracking system. 500illustrates the CT image slice. 501 illustrates a first fixation point.502 illustrates a second fixation point. 503 illustrates a thirdfixation point. 504 illustrates a fourth fixation point. 505 illustratesa fifth fixation point. 506 illustrates a sixth fixation point. 507illustrates a seventh fixation point. 508 illustrates a eighth fixationpoint. 509 illustrates a ninth fixation point. 510 illustrates a tenthfixation point. 511 illustrates a eleventh fixation point. Note that itis possible for a fixation point to be on more than one slice. Forexample, if a user is scrolling through slices rapidly, two or moreconsecutive images could have a fixation point at the same point on themonitor.

FIG. 6A illustrates a side view of the radiologist's workstation. 600illustrates a radiologist. 601 illustrates a chair that the radiologist600 is sitting on. 602 illustrates a desk that the radiologist is using.603 illustrates a monitor wherein the top portion of the monitor curvestoward the user and the bottom of the monitor curves toward the user.604 illustrates cameras, which perform eye tracking. A coordinate systemis also shown wherein the Z-direction is vertical (i.e., upward/downwarddirection towards the floor) and the X-direction is horizontal in thedirection from the user to the monitor 603.

FIG. 6B illustrates a top-down view of the radiologist's workstation.600 illustrates a radiologist. 601 illustrates a chair that theradiologist 600 is sitting on. 602 illustrates a desk that theradiologist is using. 603 illustrates a monitor wherein the left portionof the monitor curves toward the user and the right portion of themonitor curves toward the user. 604 illustrates cameras, which performeye tracking. A coordinate system is also shown wherein the Y-directionis horizontal in the left-right direction and the X-direction ishorizontal in the direction from the user to the monitor 603.

FIG. 7A is a top view of a TV/monitor screen.

FIG. 7B is a front view of the TV/monitor screen shown in FIG. 7A. Notea cross-section taken along line A. Note a cross-section taken alongline B.

FIG. 7C is a rear view of the TV/monitor screen shown in FIG. 7A.

FIG. 7D is the right side view of the TV/monitor screen shown in FIG.7A.

FIG. 7E is the left side view of the TV/monitor screen shown in FIG. 7A.

FIG. 7F is the bottom view of the TV/monitor screen shown in FIG. 7A.

FIG. 7G is a cross-sectional view taken along line A in FIG. 7B.

FIG. 7H is a cross-sectional view taken along line B in FIG. 7B. Thedevice is not limited to the scale shown herein. Also note that the top,bottom, left and right sides of the monitor can be comprised of straightedges or curved edges. The uniqueness of this design is the “doublecurved” appearance. Note that the top portion of the monitor curvesinwards towards the user. Note that the bottom portion of the monitorcurves inwards towards the user. Note that the left portion of themonitor curves inward towards the user. Note that the the right portionof the monitor curves inward towards the user. Different portions of themonitor would be roughly the same distance from the user's head. Thissolves the problem of having numerous (e.g., 8+) monitors lined up for asingle user and the monitors in the center are easily seen at the bestviewing distance and the monitors on the sides are poorly seen due tolonger viewing distances.

FIG. 8A is a view from the top of a keyboard (keys not shown) lookingdown. 800 illustrates the side of the keyboard farthest away from auser's torso 807. 801 illustrates the side of the keyboard closest to auser's torso. 802 illustrates the left side of the keyboard (i.e.,closest to where the left hand naturally types). 803 illustrates theright side of the keyboard (i.e., closest to where the right handnaturally types). 804 illustrates a cross-section through the keyboard.

FIG. 8B is a cross-section of the keyboard. This view is a cross-sectiontaken along line 804 looking from the front of the keyboard (e.g., atthe level of the user's torso). 802 illustrates the left side of thekeyboard (i.e., closest to where the left hand types). 803 illustratesthe right side of the keyboard (i.e., closest to where the right handtypes). 805 illustrates the top of the keyboard (i.e., where the keysare located). 806 illustrates the bottom of the keyboard (i.e., portionthat sits on and makes contact with the desk). Note that the middle ofthe keyboard is elevated (e.g., higher up and closer to the ceiling of aroom) as compared a side (left side 802 or right side 803). This allowsa user to strike the keys straight on with less total amount of forearmpronation. Some professions (e.g., radiologists) spent many hours a dayat a keyboard and maximizing ergonomic keyboard would therefore haveutility.

FIG. 9A illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a first location on the monitor. 900 illustrates thecomputer monitor. 901A illustrates the image on the computer monitor ata first location. Note that it does not fill up the entirety of thecomputer monitor. 902A illustrates a finding of interest (a tumor) inthe image 901A, which is fixed with respect to the image and a variablecoordinate with respect to the monitor (note that it is variable becausethe user can pan and zoom, which would cause the location of the findingof interest 902A to move to different locations on the monitor and be ofdifferent sizes on the monitor). 903A illustrates the location of thecomputer mouse, which has a variable coordinate with respect to theimage 901A (note that it is variable because the user can move it overdifferent portions of the image) and a variable coordinate with respectto the monitor 900 (note that it is variable because the user can moveit all over the monitor 900 including portions of the monitor 900 otherthan the image 901A). 904 illustrates the y-axis of the monitorcoordinate system. In this example (modeled off of the Barco Coronis5MP, which has an array of 2560×2048 pixels), the y-coordinates of themonitor coordinate system range from 1 to 2560. 905 illustrates thex-axis of the monitor coordinate system. In this example, thex-coordinates of the monitor coordinate system range from 1 to 2048. 906illustrates the y-axis of the image coordinate system. In this example,the y-coordinates of the image range from 1 to 512. 907 illustrates thex-axis of the image coordinate system. In this example, thex-coordinates of the image range from 1 to 512. Thus, the tumor 902Awould have a first set of image coordinates and a first set of monitorcoordinates.

FIG. 9B illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a second location on the monitor. Note that a pan functionhas been performed wherein the image is moved to a different location onthe monitor. This can be performed during maneuvers such to betterinspect certain portions of the image. 900 illustrates the computermonitor. 901B illustrates the image on the computer monitor at a secondlocation. Note that it does not fill up the entirety of the computermonitor. 902B illustrates a finding of interest (a tumor) in the image901B, which is fixed with respect to the image 901B and a variablecoordinate with respect to the monitor 900 (note that it is variablebecause the user can pan and zoom, which would cause the location of thefinding of interest 902B to move to different locations on the monitor900). 903B illustrates the location of the computer mouse, which has avariable coordinate with respect to the image 901B (note that it isvariable because the user can move it over different portions of theimage 901B) and a variable coordinate with respect to the monitor 900(note that it is variable because the user can move it all over themonitor 900 including portions of the monitor 900 other than the image901B). 904 illustrates the y-axis of the monitor coordinate system. Inthis example (modeled off of the Barco Coronis 5MP, which has an arrayof 2560×2048 pixels), the y-coordinates of the monitor coordinate systemrange from 1 to 2560. 905 illustrates the x-axis of the monitorcoordinate system. In this example, the x-coordinates of the monitorcoordinate system range from 1 to 2048. 906 illustrates the y-axis ofthe image coordinate system. In this example, the y-coordinates of theimage range from 1 to 512. 907 illustrates the x-axis of the imagecoordinate system. In this example, the x-coordinates of the image rangefrom 1 to 512. Thus, the tumor 902B would have the same first set ofimage coordinates (as compared to FIG. 1A), but in this example, thetumor would have a second set of monitor coordinates (different fromthat of FIG. 1A).

FIG. 10A illustrates the monitor coordinates and first coordinates foran image. 1000 illustrates the monitor. In this example, the monitor has5200 by 3400 pixels. 1001 illustrates monitor coordinate (1, 3400). 1002illustrates monitor coordinate (5200, 3400). 1003 illustrates monitorcoordinate (1, 1). 1004 illustrates monitor coordinate (5200, 1). 1005illustrates the image. In this example, the image is sized such that itis displayed as 1500 by 1500 pixels on the monitor 100. 1006 illustratesimage coordinate (1, 1500). 1007 illustrates image coordinate (1500,1500). 1008 illustrates image coordinate (1, 1). 1009 illustrates imagecoordinate (1500, 1). Note that image coordinate (1, 1) corresponds tomonitor coordinate (700, 700).

FIG. 10B illustrates the monitor coordinates and second coordinates foran image. 1000 illustrates the monitor. In this example, the monitor has5200 by 3400 pixels. 1001 illustrates monitor coordinate (1, 3400). 1002illustrates monitor coordinate (5200, 3400). 1003 illustrates monitorcoordinate (1, 1). 1004 illustrates monitor coordinate (5200, 1). 1005illustrates the image. In this example, the image is sized such that itis displayed as 1500 by 1500 pixels on the monitor 100. 1006 illustratesimage coordinate (1, 1500). 1007 illustrates image coordinate (1500,1500). 1008 illustrates image coordinate (1, 1). 1009 illustrates imagecoordinate (1500, 1). Note that image coordinate (1, 1) corresponds tomonitor coordinate (1800, 700). Note that the image has been translatedduring a pan function. The preferred embodiment is to have an eyetracking system that determines where the user is looking on the monitorand then determine where the user is looking on the image and thendetermine which structure (note that the image is segmented into variousstructures). An alternative embodiment is to track the user's eyes andthe image location directly (not specifically track the image locationon monitor location).

FIG. 11A illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a first location and a first zoom status on the monitor.1100 illustrates the computer monitor. 1101A illustrates the image onthe computer monitor at a first location and first zoom status. Notethat it fills up only a small portion of the computer monitor. 1102Aillustrates a finding of interest (a tumor) in the image 1101A, which isfixed with respect to the image and a variable coordinate with respectto the monitor (note that it is variable because the user can pan andzoom, which would cause the location of the finding of interest 1102A tomove to different locations on the monitor). 1103A illustrates thelocation of the computer mouse, which has a variable coordinate withrespect to the image 1101A (note that it is variable because the usercan move it over different portions of the image) and a variablecoordinate with respect to the monitor 1100 (note that it is variablebecause the user can move it all over the monitor 1100 includingportions of the monitor 1100 other than the image 1101A). 1104illustrates the y-axis of the monitor coordinate system. In this example(modeled off of the Barco Coronis 5MP, which has an array of 2560×2048pixels), the y-coordinates of the monitor coordinate system range from 1to 2560. 1105 illustrates the x-axis of the monitor coordinate system.In this example, the x-coordinates of the monitor coordinate systemrange from 1 to 2048. 1106 illustrates the y-axis of the imagecoordinate system. In this example, the y-coordinates of the image rangefrom 1 to 512. 1107 illustrates the x-axis of the image coordinatesystem. In this example, the x-coordinates of the image range from 1 to512. Thus, the tumor 1102A would have a first set of image coordinatesand a first set of monitor coordinates. Also, note that the tumor 1102Ahas a first size (number of pixels on the screen), which is inaccordance with the first zoom status.

FIG. 11B illustrates an example image of a monitor illustrating themonitor coordinate system and the image coordinate system wherein theimage is at a second location on the monitor. Note that a pan functionhas been performed wherein the image is moved to a different location onthe monitor. This can be performed during maneuvers such to betterinspect certain portions of the image. Also, note that the zoom functionhas been implemented, as indicated by the fact that image 1101B islarger than image 1101A. 1100 illustrates the computer monitor. 1101Billustrates the image on the computer monitor at a second location. Notethat it fills up a larger fraction of the computer monitor, as comparedto FIG. 11A. 1102B illustrates a finding of interest (a tumor) in theimage 1101B, which is fixed with respect to the image 1101B and avariable coordinate with respect to the monitor 1100 (note that it isvariable because the user can pan, which would cause the location of thefinding of interest 1102B to move to different locations on the monitor1100). 1103B illustrates the location of the computer mouse, which has avariable coordinate with respect to the image 1101B (note that it isvariable because the user can move it over different portions of theimage 101B) and a variable coordinate with respect to the monitor 1100(note that it is variable because the user can move it all over themonitor 1100 including portions of the monitor 1100 other than the image1101B). In this example, the computer mouse is located on the monitor1100, but off of the image 1101B. 1104 illustrates the y-axis of themonitor coordinate system. In this example (modeled off of the BarcoCoronis 5MP, which has an array of 2560×2048 pixels), the y-coordinatesof the monitor coordinate system range from 1 to 2560. 1105 illustratesthe x-axis of the monitor coordinate system. In this example, thex-coordinates of the monitor coordinate system range from 1 to 2048.1106 illustrates the y-axis of the image coordinate system. In thisexample, the y-coordinates of the image range from 1 to 512. 1107illustrates the x-axis of the image coordinate system. In this example,the x-coordinates of the image range from 1 to 512. Thus, the tumor1102B would have the same first set of image coordinates (as compared toFIG. 11A), but in this example, the tumor would have a second set ofmonitor coordinates (different from that of FIG. 11A).

FIG. 12 illustrates application of the conversion key to convert where auser is looking on a computer monitor to where a user is looking on animage. In this example, a pan procedure is performed. A similarprocedure can be performed for zoom. 1200 illustrates a chart for thex-coordinate conversion key with the first column showing time point (inseconds). The second column showing X monitor coordinate and the thirdcolumn showing the X image coordinate and note that together, these areused to transform the X-coordinate of the eye tracking system on themonitor into the computed X-coordinate of the eye tracking on the image.In the preferred embodiment, the image would be segmented intostructures (e.g., anatomic structures such as the radius bone, pathologystructures such as a brain aneurysm, and surgical hardware devices).This process enables real time tracking of what the user (e.g.,radiologist) is looking at in the image with associated timing metricsincluding duration and sequence. Recording the monitor coordinate of atleast one X data point within an image wherein the at least one datapoint within an image serves as a reference point for all other datapoints within the image. A zoom status is also needed to be recorded.1201 illustrates a chart for the y-coordinate conversion key with thefirst column showing time point (in seconds). The second column showingY monitor coordinate and the third column showing the X image coordinateand note that together, these are used to transform the Y-coordinate ofthe eye tracking system on the monitor into the computed Y-coordinate ofthe eye tracking on the image. In the preferred embodiment, the imagewould be segmented into structures (e.g., anatomic structures such asthe radius bone, pathology structures such as a brain aneurysm, andsurgical hardware devices). This process enables real time tracking ofwhat the user (e.g., radiologist) is looking at in the image withassociated timing metrics including duration and sequence. Recording themonitor coordinate of at least one Y data point within an image whereinthe at least one data point within an image serves as a reference pointfor all other data points within the image.

FIG. 13 illustrates generation longitudinal dataset. In this example, adataset is generated on a user's (e.g., radiologist's) analysis of anexamination. A series of variables are recorded at each time point.These variables may include, but are not limited to, the following: timepoint; a conversion key (i.e., which computer monitor coordinatecorresponds to which image coordinate with the associated zoom setting);inputs (if any) by a user (e.g., pan, zoom, window/level, other visualrepresentation adjustment logic); mouse location; and, eye trackingmetrics. Note that the eye tracking coordinate on the image can becomputed during user inputs such as panning and zooming, as previouslydescribed. Eye tracking is performed to determine a user's fixationlocation on the monitor coordinate system, which in the preferredembodiment is performed at a rapid rate (0.01 second intervals orfaster). The monitor-image conversion key is used to determine whichfixation location corresponds to which data point within the image (andthe associated imaging finding). In some embodiments, an artificialintelligence (AI) algorithm is performed on this newly generateddataset. AI processes can be performed for diagnostic purposes. Forexample, if eye tracking data shows that a human is looking at aparticular spot more times than is typical, then AI processes can befocused in at this spot (or slice). AI processes can also be performedto determine user metrics (e.g., attentiveness, search pattern adequacy,etc.). Additionally, it is important to be able to determine how eyetracking on a first scan correlates to eye tracking on a second scan.For example, consider an adrenal mass imaged in 2019 and 2020. To teachthis, it should be noted that the adrenal gland is a deformable tissue(it can change shape over time) and also a movable tissue (it can bemoved in position). The same thing is true for other soft structures inthe body, such as the kidney. During the 2019 examination, the adrenalgland is flattened from anterior to posterior and is moved (e.g., pushedto the lateral side). During a 2020 examination, the adrenal glandconfiguration is flattened from medial to lateral and is moved (e.g.,pushed to the medial side). Integrating an organ specific coordinatesystem into eye tracking is therefore useful, as disclosed in U.S.Provisional Patent Application 62/939,685, METHOD AND APPARATUS FORDEVELOPMENT OF AN ORGAN-SPECIFIC COORDINATE SYSTEM, filed on Nov. 25,2019, which is incorporated by reference. For example, it would beuseful to understand that Radiologist A spent a significant amount oftime looking at the medial limb of the left adrenal gland in 2019because that would clue in Radiologist B who is reading the scan in2020. The image could be marked (e.g., false color, arrow, circle, otherannotations) with the key spots reviewed by a prior radiologist on pastexaminations. A process to ensure that Radiologist reviews all key areasof concern can also be implemented.

FIG. 14 illustrates a method of altering display settings based oninformation from eye-tracking technology. In processing block 1400, alist of optimal viewing settings (e.g., predetermined settings) for eachitem in an image is generated. For example, an example item would be thebrain and a predetermined setting for the brain would be a brain windowof 30/30. In processing block 1401, the structure that is located ateach viewing location (e.g., pixel on 2D monitor or 3D point in spacecorresponds to liver) is determined. In processing block 1402, aneye-tracker system with an eye-facing camera(s) is initiated. Inprocessing block 1403, eye-movement data with the said eye-facingcamera(s) is recorded. In processing block 1404, analysis of theeye-movement data to determine where the user is looking (e.g., thefocal point, the fixation point) on the monitor is performed. Inprocessing block 1405, analysis of where the user is looking todetermine which item (e.g., brain) the user is examining is performed.In processing block 1406, the current image viewing settings is comparedwith optimal viewing settings for the particular item in an image beingexamined to determine whether the viewed object is optimally displayedto the user. Next, the process is illustrated to vary based on whetheror not the viewed object is optimally displayed to the user. Inprocessing block 1407, if the viewed object is already optimallydisplayed to the user, no changes to the image would be performed. Inprocessing block 1408, if the viewed object is not already optimallydisplayed to the user, viewing settings would be altered such that it isoptimally displayed (e.g., smart zoom, smart pan, smart window/level,etc.). Finally, processing block 1409 is to continue eye tracking andoptimization of image display settings, as above.

FIG. 15 illustrates an example wherein the spot at which the user islooking already has optimized viewing settings and no changes are made.In processing block 1500, analysis of the eye-movement data to determinewhere the user is looking (e.g., pixel located at row 300, column 300 ofthe 2048×1536 display) is performed. In processing block 1501, analysisof where the user is looking to determine which object the user islooking at (e.g., liver) is performed. In processing block 1502, thecurrent image viewing settings (e.g., optimized for viewing of theliver) is compared with optimal viewing settings for structure inprevious step (e.g., since the user is looking at the liver, the imagesettings should be optimized for liver) a conclusion is that viewedobject is already optimally displayed to the user. In processing block1503, no changes to the image would be performed. Note that in thisexample, the example of comparing with optimal viewing settings isillustrated. However, in practice, it could be that a new feature wouldlike to be shown based on where the user is looking. For example, anobject would not appear until the eyes look at a certain spot.Alternatively, an object would not disappear until the eyes look at adifferent spot.

FIG. 16 illustrates an example wherein the spot at which the user islooking is not currently optimized viewing settings and changes to theviewing settings are automatically performed. In processing block 1600,analysis of the eye-movement data to determine where the user is looking(e.g., pixel located at row 600, column 600 of the 2048×1536 display) isperformed. In processing block 1601, analysis of where the user islooking to determine which object the user is looking at (e.g., spleen)is performed. In processing block 1602, the current image viewingsettings (e.g., optimized for viewing of the liver) is compared withoptimal viewing settings for structure in previous step (e.g., since theuser is looking at the spleen, the image settings should be optimizedfor the spleen) and conclude that viewed object is not currentlyoptimally displayed to the user. In processing block 1603, the imagesettings from the previous settings (e.g., from being optimized forviewing of the liver) is changed to the new image settings (e.g., beingoptimized for the spleen).

FIG. 17A illustrates assigning zones wherein when a user looks at aparticular zone, a corresponding image manipulation occurs. Note that inthis embodiment, the zones which are rectangle shaped would notperfectly correspond to the anatomic structures of the human body, whichare not rectangle shaped. This boundary is meant to approximate theregions of a CT chest examination in the coronal plane at the posteriorthird of the chest. This is a simplified segmentation algorithm meant toserve as a first approximation, which will be followed by more preciseboundaries, as shown in FIG. 17B. For example, the radiologist mayprefer to have the lung display settings be displayed anytime thathe/she is looking at the top left (i.e., Zone #1) of the monitor for thedisplay settings to be optimized for lung. If his/her eyes ever soslightly gazed over to the fat within the chest wall, he/she may preferto still have the settings be optimized for lung and would not want itto change to an optimized setting for viewing fat. Similarly, theradiologist can look at Zone #2 and have image settings optimized forthe bone. Similarly, the radiologist can look at Zone #3 and have imagesettings optimized for the lung. Similarly, the radiologist can look atZone #4 and have imaging settings optimized for the liver. Similarly,the radiologist can look at Zone #5 and have imaging settings optimizedfor the spleen. Thus, a map of pixel location and preferred viewingsettings would be established per user preference. Note that in thepreferred embodiment, a 3D pixel map would be generated. For eachviewing location (e.g., pixel on 2D monitor), determine the structurethat is located at that point (or in that region) and then optimize theimage for the anatomic structure in that region. A double windowingtechnique may be used in conjunction with this, as described in U.S.Pat. No. 10,586,400. This will help prevent some errors. For example,sometimes a radiologist is looking and examining bone for quite sometime yet the window/level settings are optimized for soft tissue andcould miss a finding as a result. This system would resolve thispotential source of error.

FIG. 17B illustrates dividing up the field of view into regions based onsegmentation. This method of dividing may be more closely aligned withsegmentation algorithms. For example, if the user is looking at the areadefined by the liver region 1700, then a liver window is displayed asshown in 1701. If the user is looking at the area defined by the rightlung base 1702, then a lung window is displayed as shown in 1703. Adouble windowing technique may be used in conjunction with this, asdescribed in U.S. Pat. No. 10,586,400, PROCESSING 3D MEDICAL IMAGES TOENHANCE VISUALIZATION, which is incorporated by reference. In addition,the zones could be determined by methods described in U.S. patentapplication Ser. No. 16/785,606, IMPROVING IMAGE PROCESSING VIA AMODIFIED SEGMENTED STRUCTURE, which is incorporated by reference.

FIG. 18 illustrates generating a list of the optimal viewing settingsfor each item in an image. 1800 illustrates a chart showing two itemswithin an image are shown along with the optimal viewing settings during2D slice-by-slice viewing per user preference. For example, liver couldbe shaded as a rainbow color scheme to bring out subtle lesions. Allother tissues in the image slice are turned to dark gray shades, whichwill provide context. For example, bone is colored ranging from mediumgray to very light gray shades (per user preference). All other tissuesare turned to dark gray shades, which will provide continued context.1801 illustrates a chart showing two items within an image are shownalong with the optimal viewing settings during 3D augmented realityvolume-by-volume viewing per user preference. For example, bands-wiseprioritization of HU ranges is utilized within the liver and displayedin a dynamic fashion to make more subtle (but dangerous lesions) easierto detect. To perform this, the voxels that subtend the liver aredivided into bands based on their property (e.g., Hounsfield Unit). Forexample, assume that voxels that subtend the liver have Hounsfield of30-60. These can be divided into 3 bands (e.g., upper range of 50-60 HU,a middle range of 40-50 HU, and a lower range of 30-40 HU) wherein atthree different time points one of the bands has enhanced visualization(e.g., grayscale) and the other two bands have diminished visualization(e.g., black). This process wherein voxels are divided into bands andthen visualization enhanced or diminished improves detection of subtlelesions. All other tissues are made more translucent (e.g., sparsesampling) or are filtered. For example, for bone prioritized volumerendering (U.S. patent Ser. No. 16/842,631) is performed wherein thebone surface is displayed unless the is a lesion within the centralaspect of the bone, which would then be higher priority and bedisplayed. All other tissues are made more translucent (e.g., sparsesampling) or are filtered. This is more thoroughly described in U.S.Pat. No. 10,586,400, the figures and detailed description.

FIG. 19A illustrates an image optimized for visualization of abdominalorgans. 1900 illustrates a CT slice where the entire CT slice isdisplayed with a single window/level setting.

FIG. 19B illustrates an image optimized for visualization of abdominalorgans with a fixation point. 1900 illustrates a CT slice where theentire CT slice is displayed with a single window/level setting. 1901illustrates a first fixation point (determined by the eye trackingsystem), which is located on the liver.

FIG. 19C illustrates an image, which has been altered due to a priorfixation point. 1902 illustrates a CT slice where a portion of the imagehas been altered (darkened in this example) to indicate that it has beenreviewed. The area darkened can be determined by user preference. 1903illustrates the darkened portion of the image from the first fixationpoint. Note that the remaining portions of the image are shown withnormal brightness, as in FIGS. 19A and 19B.

FIG. 19D illustrates an image, which is partially darkened with a secondfixation point shown. 1902 illustrates a CT slice where a portion of theimage has been altered (darkened in this example) to indicate that ithas been reviewed. 1903 illustrates the darkened portion of the image.Note that the remaining portions of the image are shown with normalbrightness, as in FIGS. 19A and 19B. 1904 illustrates a second fixationpoint, which is shown on the right kidney.

FIG. 19E illustrates an image, which is partially darkened at thelocations of the two prior fixation points. 1905 illustrates a CT slicewhere a portion of the image has been altered (darkened in this example)to indicate that it has been reviewed. 1903 illustrates the darkenedportion of the image from the first fixation point. 1906 illustrates thedarkened portion of the image from the second fixation point.

FIG. 19F illustrates an image, which is completely darkened, whichoccurs when a slice is fully inspected. 1907 illustrates a CT slicefully darkened, which indicates that is has been completely reviewed.Thus, this embodiment provides a process of changing the appearance ofan image based on eye tracking. The visual appearance can change interms of brightness (darkening vs. brightening), contrast (sharp vs.blurred), timing of display (during inspection vs after inspection),rate of display of the new visual appearance (rapid display of new imagevs fading in of new image over time). Note that the darkening can beshown over multiple time steps (in accordance with the fixation points)or the user could be allowed a period of time (e.g., 4.0 seconds) andthen all areas closely inspected with fixation points darkened and allareas not closely inspected with fixation points shown in normalbrightness fashion. Furthermore, areas that are not reviewed could beflagged to the user. Overall, these processes improves image analysis byalerting to the reviewer which areas have been inspected and which areashave not been inspected. Furthermore, in some embodiments, areasactively under inspection could be given an first visual representationadjustment logic. Areas that have been previously inspected could begiven a second visual representation adjustment logic. Areas that havenot been inspected could be given a third visual representationadjustment logic.

FIG. 20 illustrates an example of changing of the appearance of an imagein accordance with eye tracking. 2000 illustrates a processing block ofdisplaying an image with a first example set of parameters (e.g., softtissue window). An example includes a CT slice through the abdomen witha standard window and level setting (e.g., soft tissue window). 2001illustrates a processing block of moving to a segmented item to beanalyzed (e.g., via eye tracking is performed and it is determined thatthe user is closely inspecting the pancreas, etc.). The pancreastherefore acts as a triggering spot. 2002 illustrates a processing blockof adjusting display settings (e.g., window and leveling) of an item(e.g., pancreas) to be analyzed to optimize viewing of the item andoption to also adjust display settings of segmented items not currentlybeing analyzed. In this example, the double windowing techniquedescribed in U.S. Pat. No. 10,586,400, PROCESSING 3D MEDICAL IMAGES TOENHANCE VISUALIZATION, was performed, which allows improved visualappearance of the pancreas. An alteration of the appearance of an imagebased on eye tracking and where the user is looking at on the image isperformed. In this example, dual windowing is shown wherein a firstportion of the image (i.e., pancreas) is shown with optimum window andlevel setting and the remainder of the image is shown with bonewindow/level setting. This serves to bring the user's attention to thepancreas. Other techniques, such as halo windowing can also beincorporated. In addition, with regards to changing display settings, ifthe user is looking at the vertebral body for more then 1.00 seconds,the visual representation can be set to automatically change to optimizeviewing of the vertebral body (e.g., optimize gray scale appearance ofthe vertebral body and make all other structures in the field of viewdarkened). A wide range of visual representation adjustment logicschemes are anticipated to be performed in response to eye trackingmetrics. First, techniques include voxel filtering and stereoscopicrendering and others are incorporated as described by U.S. Pat. No.8,384,771, METHOD AND APPARATUS FOR THREE DIMENSIONAL VIEWING OF IMAGES,which is incorporated by reference. Next, techniques include convergenceand others are incorporated as described by U.S. Pat. No. 9,349,183,METHOD AND APPARATUS FOR THREE DIMENSIONAL VIEWING OF IMAGES, which isincorporated by reference. Next, techniques include the use ofalternative head display units and others are incorporated as describedby U.S. Pat. No. 9,473,766, METHOD AND APPARATUS FOR THREE DIMENSIONALVIEWING OF IMAGES, which is incorporated by reference. Next, techniquesinclude the use of a 3D volume cursor and others are incorporated asdescribed by U.S. Pat. No. 9,980,691, METHOD AND APPARATUS FOR THREEDIMENSIONAL VIEWING OF IMAGES, which is incorporated by reference. Next,techniques include the use of an interactive 3D cursor and others areincorporated as described by U.S. patent application Ser. No.15/878,463, INTERACTIVE 3D CURSOR FOR USE IN MEDICAL IMAGING, which isincorporated by reference. Next, techniques include double windowing andothers are incorporated and others as described in U.S. Pat. No.10,586,400, PROCESSING 3D MEDICAL IMAGES TO ENHANCE VISUALIZATION, whichis incorporated by reference. Next, techniques including use of modifiedsegmented structure and others are incorporated as described in U.S.patent application Ser. No. 16/785,606, IMPROVING IMAGE PROCESSING VIA AMODIFIED SEGMENTED STRUCTURE, which is incorporated by reference. Next,techniques including use of double compression mammography and othersare incorporated as described in U.S. patent application Ser. No.16/594,139, METHOD AND APPARATUS FOR PERFORMING 3D IMAGING EXAMINATIONSOF A STRUCTURE UNDER DIFFERING CONFIGURATIONS AND ANALYZING MORPHOLOGICCHANGES, which is incorporated by reference. Next, techniques includingthose of smart scrolling and others are incorporated as described inU.S. patent application Ser. No. 16/842,631, A SMART SCROLLING SYSTEM,which is incorporated by reference. Next, techniques of eye tracking areincorporated as disclosed in U.S. Provisional Patent Applications62/856,185 filed on Jun. 3, 2019 and 62/985,363 filed on Mar. 5, 1920,which are incorporated by reference. Next, techniques of affixing asub-volume onto a geo-registered tool are incorporated as disclosed inU.S. Pat. No. 10,712,837, USING GEO-REGISTERED TOOLS TO MANIPULATETHREE-DIMENSIONAL MEDICAL IMAGES, which is incorporated by reference.Next, techniques of virtual toolkit and others are incorporated asdisclosed in PCT/US2019/036904, A VIRTUAL TOOL KIT FOR 3D IMAGING, whichis incorporated by reference. Next, techniques of interaction betweengeo-registered tools and virtual tools are incorporated as disclosed inU.S. patent application Ser. No. 16/563,985, A METHOD AND APPARATUS FORTHE INTERACTION OF VIRTUAL TOOLS AND GEO-REGISTERED TOOLS, which isincorporated by reference. Next, techniques of prioritized volumerendering are incorporated as disclosed in U.S. patent application Ser.No. 16/879,758, A METHOD AND APPARATUS FOR PRIORITIZED VOLUME RENDERING,which is incorporated by reference. Next, techniques of radiologistassisted machine learning are incorporated as disclosed inPCT/US2019/023968, RADIOLOGIST-ASSISTED MACHINE LEARNING WITHINTERACTIVE, VOLUME-SUBTENDING 3D CURSOR, which is incorporated byreference. Next, techniques of illustrating flow are incorporated asdisclosed in U.S. patent application Ser. No. 16/506,073, A METHOD FORILLUSTRATING DIRECTION OF BLOOD FLOW VIA POINTERS, and Ser. No.16/779,658, 3D IMAGING OF VIRTUAL FLUIDS AND VIRTUAL SOUNDS, which areincorporated by reference. Next, techniques of sub-volume isolation andtargeting are incorporated as disclosed in U.S. patent application Ser.No. 16/927,886, A METHOD AND APPARATUS FOR GENERATING A PRECISIONSUB-VOLUME WITHIN THREE-DIMENSIONAL IMAGE DATASETS, which isincorporated by reference.

FIG. 21 illustrates a smart zoom process. 2100 illustrates a processingblock of determining the optimum angular resolution for a user. 2101illustrates a processing block of performing segmentation of the image.2102 illustrates a processing block of moving to a segmented item (e.g.,via user input, via eye tracking, etc.). 2103 illustrates a processingblock of determining the optimum display size of the item on thechecklist. 2104 illustrates a processing block of automatically re-sizethe image such that item on the checklist via zooming (e.g., may alsouser's viewing point). Subsequently, return to processing block 2102.

FIG. 22A illustrates an imaging finding displayed on a monitor. 2200illustrates a point between the eyes. 2201 illustrates the monitor. α₁illustrates the angular resolution that object D₁ appears on the screen.L₁ illustrates the distance from the point between the eyes 2200 to themonitor 2201. Assume a 30 inch monitor. Assume viewing distance of 24in. Assume an adrenal gland shows up at a 0.5 inch item on the screenunder a first viewing setting. The angular resolution of the adrenalgland would be approximately 1.2 degrees.

FIG. 22B illustrates an imaging finding displayed on a monitor. 2200illustrates a point between the eyes. 2201 illustrates the monitor. α₂illustrates the angular resolution that object D₂ appears on the screen.L₂ illustrates the distance from the point between the eyes 2200 to themonitor 2201. Assume a 30 inch monitor. Assume viewing distance of 24in. Assume that the optimum angular resolution of the adrenal gland is2.4 degrees. Once implemented (e.g., via moving to the adrenal glanditem on the checklist or via eye tracking wherein the user looks at theadrenal gland for some pre-specified time period such as 1.00 seconds),the adrenal gland is enlarged on the monitor up to a size of 1.0 inch.This enlargement can occur in an instant (over a single frame) or viagradual enlargement (over several frames). The “smart zoom” functionenlarges the size of the adrenal gland on the screen to the desiredlevel (e.g., 2 degrees, 3 degrees, etc.). This can improve imagedetection and analysis. In this embodiment, a smart zoom process isinitiated. Some anatomic structures, such at the adrenal gland, arerelatively small. Small nodules, such as adrenal adenomas, would bebetter detected if the adrenal glands were displayed on the radiologymonitor in an enlarged fashion. The preferred method for a smart zoomprocess comprises showing an anatomic feature at a size on the monitor,which is optimized for the user. A user's fovea has the optimum visualacuity and is typically approximately 2 degrees. There would be littleutility to show a structure smaller than 2 degrees and the user's highresolution visual acuity would not be fully used. As the distance awayfrom the fovea increased the visual acuity drops. Thus, for someanatomic features (e.g., adrenal gland) wherein very close inspection isnecessary, a smart zoom automatically sizes the anatomic feature (e.g.,adrenal gland) appropriately. For example, the adrenal gland can take onthe appropriate size on the monitor that the anatomic structure appears2× the fovea field of view or approximately 4 degrees. Note that thetypical state of the right adrenal gland. In some embodiments, the smartzoom would automatically size an anatomic feature on the image for 1.5×the fovea field of view, which would be 3 degrees. In some embodiments,the smart zoom would automatically size an anatomic feature on the imagefor 2× the fovea field of view, which would be 4 degrees. In someembodiments, the smart zoom would automatically size an anatomic featureon the image for 3× the fovea field of view, which would be 6 degrees.In some embodiments, the smart zoom would automatically size an anatomicfeature on the image for a user-specified multiplier of the fovea fieldof view. And so on. The preferred embodiment of this process is toperformed smart zoom in conjunction with a radiologist's checklist. Forexample, the radiologist's first item on an abdominal CT scan checklistis the liver. In accordance with the pre-determined optimal zoom status,the liver is at 7.5× the fovea field of view, which would be 15 degrees.The CT scan slices of the liver are shown to the user in a first zoomstate wherein the liver comprises 15 degrees of the user's field ofview. Once the radiologist has completed the liver item on thechecklist, the radiologist moves to the gallbladder, which is the seconditem on the checklist. In accordance with the pre-determined optimalzoom status, the gallbladder is at 2× the fovea field of view, whichwould be 4 degrees. The CT scan slices of the gallbladder are shown tothe user in a second zoom state wherein the gallbladder comprises 4degrees of the user's field of view. Once the radiologist has completedthe gallbladder item on the checklist, the radiologist moves to thespleen, which is the third item on the checklist. In accordance with thepre-determined optimal zoom status, the spleen is at 4 x the fovea fieldof view, which would be 8 degrees. The CT scan slices of the spleen areshown to the user in a third zoom state wherein the spleen comprises 4degrees of the user's field of view. Once the radiologist has completedthe spleen item on the checklist, the radiologist moves to the pancreas,which is the fourth item on the checklist. In accordance with thepre-determined optimal zoom status, the pancreas is at 3.5× the foveafield of view, which would be 7 degrees. The CT scan slices of thepancreas are shown to the user in a fourth zoom state wherein thepancreas comprises 7 degrees of the user's field of view. Once theradiologist has completed the pancreas item on the checklist, theradiologist moves to the right adrenal gland, which is the fifth item onthe checklist. In accordance with the pre-determined optimal zoomstatus, the right adrenal gland is at 2× the fovea field of view, whichwould be 4 degrees. The CT scan slices of the right adrenal gland areshown to the user in a fifth zoom state wherein the right adrenal glandcomprises 4 degrees of the user's field of view.

FIG. 23A illustrates determining and recording which areas (e.g., voxelsor pixels) are included in the high resolution FOV, which areas (e.g.,voxels or pixels) are included in the medium resolution FOV, and whichareas (e.g., voxels or pixels) are included in the low resolution FOV. Atable is illustrates to show data that can also be collected in alongitudinal fashion. In this embodiment, three sets of voxels areincluded at each time point. During time point 0.01 seconds, set A ofvoxels would be recorded in the high resolution field of view, set B ofvoxels would be recorded in the medium resolution field of view, and setC of voxels would be recorded in the low resolution field of view.During time point 0.02 seconds, set D of voxels would be recorded in thehigh resolution field of view, set E of voxels would be recorded in themedium resolution field of view, and set F of voxels would be recordedin the low resolution field of view. During time point 0.03 seconds, setG of voxels would be recorded in the high resolution field of view, setH of voxels would be recorded in the medium resolution field of view,and set I of voxels would be recorded in the low resolution field ofview. During time point 0.04 seconds, set J of voxels would be recordedin the high resolution field of view, set K of voxels would be recordedin the medium resolution field of view, and set L of voxels would berecorded in the low resolution field of view.

FIG. 23B illustrates a zoomed in 8×8 set of voxels, showing assigningsome voxels to a high resolution FOV and some voxels to a mediumresolution FOV in accordance with FIG. 23A. A total of 24 voxels areillustrated in Set A in this example. Set B would contain an additional40 voxels in this example. Voxels assigned to Set C are not shown inthis Figure. The voxels could be stored by their (x, y, z locations).This would be useful because it would allow a more precise way ofdetermining how well the 3D dataset has been reviewed. Furthermore, itis useful because voxel located at data point at the x, y, z coordinate(150, 200, 250) could be tracked at which time points it is viewedduring the examination. Furthermore, voxels that are marked as abnormalby an AI algorithm could also be tracked and determined how well theseareas have been reviewed by a human (e.g., radiologist).

FIG. 24 illustrates a method of generating metrics based on whichimaging features have been reviewed. Processing block 2400 determinesmetrics (e.g., imaging features thoroughly viewed, imaging features notexamined). Processing block 2401 displays an imaging dataset on acomputer monitor. Processing block 2402 performs segmentation of theimaging dataset (e.g., segment a brain MRI into the frontal lobe,temporal lobe, pituitary gland, etc.). Processing block 2403 determinesthe location(s) of imaging features on the monitor (e.g., performed in adynamic fashion wherein an imaging finding may change in position overtime during zooming or panning by a user). Processing block 2404 trackseye movements of a user to determine the fixation locations at pixels onthe monitor and the corresponding imaging features being viewed.Processing block 2405 records data on fixation locations and discreteimaging features (e.g., sequence of viewing of imaging features, lengthof time an imaging feature has been viewed, etc.). Processing block 2406analyzes the recorded data on fixation locations and discrete imagingfeatures. Processing block 2407 reports metrics to the user. Processingblock 2408 alters imaging display based on metrics above (e.g., makeimaging features not sufficiently viewed based on predeterminedstandards stand out in such a way as to draw the user's attention).

FIG. 25 illustrates assigning a set of predetermined locations within animage that should be viewed by a user in order for a comprehensivereview to be performed. 2500 illustrates a processing block ofperforming eye-tracking with an eye-facing camera to determine a set offixation locations in the monitor. 2501 illustrates a processing blockof correlating the fixation locations to their corresponding imagingfeatures. 2502 illustrates a processing block of determining whichpredetermined locations have been viewed and which predeterminedlocations have not been viewed. 2503 illustrates a processing block ofalerting user of those predetermined locations which have not beenviewed. A first example is providing a visual alert cue adjacent tothose predetermined locations which have not been viewed. A secondexample is providing a first visual representation adjustment logic forthe pixels nearby the predetermined locations which have been viewed anda second visual representation adjustment logic for those pixels nearbythe predetermined locations which have not been viewed.

FIG. 26 illustrates comparing the analyzed recorded data on fixationlocations and discrete imaging features with a predetermined criteria ofminimum imaging metrics that must be met for a complete review.Processing block 2600 illustrates a text box of example criteria aminimum number of fixation locations for the imaging dataset. Thesecriteria include: a minimum number of fixation locations for eachimaging feature (e.g., 5 fixation locations within 10 mm of the centralpoint of the pituitary gland); a minimum number of fixation locationsfor each subsegmented spot (e.g., frontal lobe) within an imagingfeature (e.g., brain); a minimum length of fixation location for eachimaging feature (e.g., 50 fixation locations for the liver); a minimumnumber of imaging planes an imaging feature has fixation locations insituations comprising wherein the imaging dataset comprisescross-sectional imaging planes (e.g., fixation locations on 3 plans forthe pituitary gland, fixation locations on two planes for the corpuscallosum, etc.); whether or not the imaging structure had optimizeddisplay during a fixation location (e.g., a fixation location on thevertebral body is considered adequate if the vertebral body is windowedso that the vertebral body is optimized whereas a fixation location onthe vertebral body is not considered adequate if the vertebral body isnot windowed so that the vertebral body is optimized); and, whether ornot a predetermined sequence of fixation locations for each imagingfeature has been achieved (e.g., predetermined sequence would be aorticarch, lower common carotid artery, middle common carotid artery, uppercommon carotid artery, lower internal carotid artery, middle internalcarotid artery and upper internal carotid artery, which shows that amethodical search was performed. Random points along the carotid arterymay not indicate as comprehensive of a search.). Processing block 2601illustrates a text box of altering the displayed image based on therelationship between the analyzed data and the predetermined threshold.Processing block 2602 illustrates a text box of altering the brightnessof imaging feature(s) that have met the predetermined threshold whereinthe imaging feature(s) that have met the predetermined threshold areassigned a first visual representation adjustment logic. Processingblock 2603 illustrates a text box of altering the brightness of imagingfeature(s) that have not met the predetermined threshold wherein theimaging feature(s) that have not met the predetermined threshold areassigned a second visual representation adjustment logic.

What is claimed is:
 1. A method comprising: for a first time epoch,performing the steps comprising: displaying an image onto a display witha first set of display settings; using an eye facing camera to track auser's first fixation location on the display; determining a firstimaging finding located at the user's first fixation location on thedisplay; recording data in a dataset including the first imaging findingwith a first set of display settings; for a subsequent time epoch,performing the steps comprising: displaying the image with a subsequentset of display settings wherein the subsequent set of display settingsis different from the first set of display settings; using the eyefacing camera to track the user's subsequent fixation location on thedisplay; determining a subsequent imaging finding located at the user'ssubsequent fixation location on the display wherein the subsequentimaging finding comprises a segmented item, wherein the segmented itemis assigned an optimal viewing setting, and wherein the subsequentfixation location at the subsequent imaging finding causes thesubsequent imaging finding to be displayed with said optimal viewingsetting; and recording data in the dataset including the subsequentimaging finding with the subsequent set of display settings.
 2. Themethod of claim 1 further comprising analyzing the dataset to determinethe extent of the review by the user.
 3. The method of claim 1 furthercomprising analyzing the dataset comprising generating metricscorrelating fixation locations with imaging findings comprises of atleast one of the group consisting of: a number of fixation location(s)for an imaging finding; an imaging plane of fixation location(s) for animaging finding; and a sequence of fixation location(s) for an imagingfinding.
 4. The method of claim 3 further comprising determining anaverage for each metric for the user and providing an notification tothe user when a current exam's metric differs from the average.
 5. Themethod of claim 3 further comprising determining an average for eachmetric for a population of users to develop a set of normative metricsand providing a notification to the user when the user's metric differsfrom the population.
 6. The method of claim 1 further comprisingcomparing the dataset with a predetermined criteria to determine whethera minimum review has been completed wherein the predetermined criteriacomprises of at least one of the group consisting of: a minimum numberof fixation locations for the image; a minimum number of fixationlocations for each imaging finding; a minimum number of fixationlocations for each subsegmented area within an imaging finding; aminimum viewing time for each imaging feature; a minimum number ofimaging planes an imaging finding has fixation locations in situationscomprising wherein the imaging dataset comprises cross-sectional imagingplanes; a determination of whether or not a structure had optimizeddisplay during a fixation location; and a determination of whether ornot a predetermined sequence of fixation locations for each imagingfeature has been achieved.
 7. The method of claim 6 further comprisingaltering the displayed image based on the relationship between theanalyzed dataset and the predetermined criteria further comprising:assigning a first visual representation adjustment logic to imagingfinding(s) that have met the predetermined criteria; and assigning asecond visual representation adjustment logic to imaging finding(s) thathave not met the predetermined criteria wherein the first visualrepresentation adjustment logic is different from the second visualrepresentation adjustment logic.
 8. The method of claim 7 furthercomprising providing visual feedback to the user based on whether or notthe predetermined criteria has been met to guide the user in theperformance of a comprehensive review of the image.
 9. The method ofclaim 1 further comprising wherein the display has a curvaturecomprising: the top portion of the monitor curves inwards towards theuser; the bottom portion of the monitor curves inwards towards the user;the left portion of the monitor curves inward towards the user; and theright portion of the monitor curves inward towards the user.
 10. Themethod of claim 1 further comprising utilizing a head display unit (HDU)comprising of at least one of the group consisting of: an extendedreality display; eye display with shutters; and, eye display withpolarized lenses.
 11. The method of claim 1 further comprising using akeyboard comprising wherein the center of the keyboard is in an elevatedposition with respect to the sides of the keyboard.
 12. The method ofclaim 1 further comprising wherein the display setting comprises atleast one of the group consisting of: a pan setting; a zoom setting;and, a window/level setting.
 13. The method of claim 1 further whereinthe dataset comprises data on areas of the image included in the user'shigh resolution field of view.
 14. The method of claim 1 furthercomprising using eye tracking data to implement at least one of thegroup comprising: a smart panning viewing option; a smart zoom viewingoption; and a smart window/level viewing option.
 15. The method of claim1 further comprising analyzing the dataset by an artificial intelligencealgorithm.
 16. A non-transitory computer readable medium having computerreadable code thereon for image processing, the medium comprising:instructions for a first time epoch, performing the steps comprising:displaying an image onto a display with a first set of display settings;using an eye facing camera to track a user's first fixation location onthe display; determining a first imaging finding located at the user'sfirst fixation location on the display; recording data in a datasetincluding the first imaging finding with a first set of displaysettings; instructions for a subsequent time epoch, performing the stepscomprising: displaying the image with a subsequent set of displaysettings wherein the subsequent set of display settings is differentfrom the first set of display settings; using the eye facing camera totrack the user's subsequent fixation location on the display;determining a subsequent imaging finding located at the user'ssubsequent fixation location on the display; and recording data in thedataset including the subsequent imaging finding with the subsequent setof display settings; instructions that compare the dataset with apredetermined criteria to determine whether a minimum review has beencompleted wherein the predetermined criteria comprises of at least oneof the group consisting of: a minimum number of fixation locations forthe image; a minimum number of fixation locations for each imagingfinding; a minimum number of fixation locations for each subsegmentedarea within an imaging finding; a minimum viewing time for each imagingfinding; a minimum number of imaging planes an imaging finding hasfixation locations in situations comprising wherein the imaging datasetcomprises cross-sectional imaging planes; a determination of whether ornot a structure had optimized display during a fixation location; and adetermination of whether or not a predetermined sequence of fixationlocations for each imaging feature has been achieved, instructions thatalter the displayed image based on the relationship between the analyzeddataset and the predetermined criteria further comprising: assigning afirst visual representation adjustment logic to imaging finding(s) thathave met the predetermined criteria; and assigning a second visualrepresentation adjustment logic to imaging finding(s) that have not metthe predetermined criteria where the first visual representationadjustment logic is different from the second visual representationadjustment logic; and instructions that provide visual feedback to theuser based on whether or not the predetermined criteria has been met toguide the user in the performance of a comprehensive review of theimage.
 17. A computer system comprising: a memory; a processor; andwherein the memory is encoded with an application providing imageprocessing that when performed on the processor provides a process forprocessing information, the process causing the computer system toperform the operations of: for a first time epoch, performing the stepscomprising: displaying an image onto a display with a first set ofdisplay settings; using an eye facing camera to track a user's firstfixation location on the display; determining a first imaging findinglocated at the user's first fixation location on the display; recordingdata in a dataset including the first imaging finding with a first setof display settings; for a subsequent time epoch, performing the stepscomprising: displaying the image with a subsequent set of displaysettings wherein the subsequent set of display settings is differentfrom the first set of display settings; using the eye facing camera totrack the user's subsequent fixation location on the display;determining a subsequent imaging finding located at the user'ssubsequent fixation location on the display; and recording data in thedataset including the subsequent imaging finding with the subsequent setof display settings comparing the dataset with a predetermined criteriato determine whether a minimum review has been completed wherein thepredetermined criteria comprises of at least one of the group consistingof, a minimum number of fixation locations for the image; a minimumnumber of fixation locations for each imaging finding; a minimum numberof fixation locations for each subsegmented area within an imagingfinding; a minimum viewing time for each imaging finding; a minimumnumber of imaging planes an imaging finding has fixation locations insituations comprising wherein the imaging dataset comprisescross-sectional imaging planes; a determination of whether or not astructure had optimized display during a fixation location; and adetermination of whether or not a predetermined sequence of fixationlocations for each imaging feature has been achieved; and instructionsthat alter the displayed image based on the relationship between theanalyzed dataset and the predetermined criteria further comprising:assigning a first visual representation adjustment logic to imagingfinding(s) that have met the predetermined criteria, and assigning asecond visual representation adjustment logic to imaging finding(s) thathave not met the predetermined criteria wherein the first visualrepresentation adjustment logic is different from the second visualrepresentation adjustment logic.